Skip to content

ffi: expose array validity, allow creating primitive arrays#7148

Merged
myrrc merged 1 commit intodevelopfrom
myrrc/primitive-ffi
Apr 1, 2026
Merged

ffi: expose array validity, allow creating primitive arrays#7148
myrrc merged 1 commit intodevelopfrom
myrrc/primitive-ffi

Conversation

@myrrc
Copy link
Copy Markdown
Contributor

@myrrc myrrc commented Mar 24, 2026

  • Allow querying array validity from FFI.
  • Allow creating null and primitive vx_arrays from FFI.
  • Expose utility functions to query dtype and ptype from array
    directly instead of array -> dtype -> variant [-> ptype] chain.

Breaking changes:

  • rename vx_array_is_null to vx_array_element_is_invalid.
  • rename vx_array_null_count to vx_array_invalid_count.
  • make functions operating on indices take size_t instead of u32.

@myrrc myrrc force-pushed the myrrc/primitive-ffi branch from b7d2f79 to 516884b Compare March 24, 2026 17:18
@myrrc myrrc requested review from 0ax1 and joseph-isaacs March 24, 2026 17:18
@myrrc myrrc added changelog/feature A new feature changelog/break A breaking API change labels Mar 24, 2026
@myrrc myrrc force-pushed the myrrc/primitive-ffi branch from 516884b to 430ec1e Compare March 24, 2026 17:20
@myrrc myrrc removed the changelog/feature A new feature label Mar 24, 2026
@myrrc myrrc force-pushed the myrrc/primitive-ffi branch from 430ec1e to a3c15ad Compare March 24, 2026 17:23
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq bot commented Mar 24, 2026

Merging this PR will not alter performance

✅ 1106 untouched benchmarks
⏩ 1522 skipped benchmarks1


Comparing myrrc/primitive-ffi (b8e9b2a) with develop (a3f290c)

Open in CodSpeed

Footnotes

  1. 1522 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@myrrc myrrc enabled auto-merge (squash) March 24, 2026 17:28
Copy link
Copy Markdown
Contributor

@0ax1 0ax1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to double check the consistency of size types for indices:

vx_array_get_field` | `uint32_t` |
| `vx_array_slice` | `uint32_t` |
| `vx_array_get_u8` etc. | `uint32_t` |
| `vx_array_null_count` return | `uint32_t

should prob all be size_t.

@myrrc myrrc removed the request for review from joseph-isaacs March 26, 2026 10:57
@myrrc myrrc force-pushed the myrrc/primitive-ffi branch from a3c15ad to 353e971 Compare March 27, 2026 16:42
myrrc added a commit that referenced this pull request Mar 30, 2026
Expose a subset of expressions to test filtering in Scan API:
root, column, get_item, select, or, not, is_null, and binary operators,
including shortcuts like eq.
Add vx_array_apply to test C-side expression application.

This patch does not include C++ FFI tests because we can't create
vx_arrays' yet, #7148.
The next steps are exposing literal expressions and dynamic expressions.

Signed-off-by: Mikhail Kot <mikhail@spiraldb.com>
@myrrc myrrc requested a review from 0ax1 March 31, 2026 14:15
@myrrc myrrc force-pushed the myrrc/primitive-ffi branch 5 times, most recently from 7b4b232 to e564354 Compare March 31, 2026 14:24
@myrrc myrrc force-pushed the myrrc/primitive-ffi branch 2 times, most recently from 2f7fff8 to 69de977 Compare April 1, 2026 14:25
@myrrc myrrc requested a review from 0ax1 April 1, 2026 14:25
@myrrc myrrc force-pushed the myrrc/primitive-ffi branch from 69de977 to e18ed69 Compare April 1, 2026 14:27
@myrrc myrrc force-pushed the myrrc/primitive-ffi branch from e18ed69 to 58228bb Compare April 1, 2026 14:29
Signed-off-by: Mikhail Kot <mikhail@spiraldb.com>
Signed-off-by: Mikhail Kot <to@myrrc.dev>
@myrrc myrrc merged commit a6cc251 into develop Apr 1, 2026
58 checks passed
@myrrc myrrc deleted the myrrc/primitive-ffi branch April 1, 2026 15:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/break A breaking API change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants